Rearview vision system with indicia of backup travel

ABSTRACT

A rearview vision system for a vehicle includes an image capture device mounted at the rear of the vehicle and having a field of view directed rearwardly of the vehicle. A display system is viewable by a driver of the vehicle. The display system displays a rearward image output of the image capture device. A graphic overlay is superimposed on the rearward image when the gear actuator of the vehicle selects a reverse gear. The graphic overlay may include indicia of the anticipated path of travel of the vehicle. The graphic overlay may be disabled when the gear actuator of the vehicle is not in reverse gear.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of application Ser. No. 08/935,336,filed on Sep. 22, 1997, which is a continuation of application Ser. No.08/445,527, filed on May 22, 1995, now U.S. Pat. No. 5,670,935, which isa continuation-in-part of patent application Ser. No. 08/023,918 filedFeb. 26, 1993, by Kenneth Schofield and Mark Larson now U.S. Pat. No.5,550,677.

BACKGROUND OF THE INVENTION

This invention relates generally to vision systems for vehicles and,more particularly, to rearview vision systems which provide the vehicleoperator with scenic information in the direction rearward of thevehicle. More particularly, the invention relates to a rearview visionsystem utilizing image capture devices, such as CMOS imaging arrays andthe like.

A long-felt need in the art of vehicle rearview vision systems has beento eliminate exterior rearview mirrors by utilizing image capturedevices, such as cameras, in combination with dashboard displays. Thiswould be beneficial because it would reduce wind drag on the vehicle,wind noise and vehicle weight. Furthermore, rearview mirrors protrude asubstantial distance from the side of the vehicle, which makesmaneuvering in tight spaces more difficult. Image capture devices arecapable of positioning in a greater variety of locations on the vehicle,providing more flexibility of vehicle styling. It is further expectedthat camera systems would greatly reduce the blind spots to the sidesand rear of the vehicle common with vehicles equipped with conventionalrearview mirror systems. The driver cannot perceive vehicles, objects,or other road users in such blind spots without turning his or her body,which interferes with forward-looking visual activities.

Camera-based rearview vision systems for vehicles have not obtainedcommercial acceptance. One difficulty with proposed systems has beenthat they present a large amount of visual information in a manner whichis difficult to comprehend. This difficulty arises from many factors. Inorder to significantly reduce blind spots, multiple image capturedevices are typically positioned at various locations on the vehicle.The image of an object behind the equipped vehicle is usually capturedby more than one image capture device at a time and displayed inmultiple images. This may confuse the driver as to whether more than oneobject is present. When multiple image capture devices are positioned atdifferent longitudinal locations on the vehicle, objects behind thevehicle are at different distances from the image capture devices. Thisresults in different image sizes for the same object. This effect isespecially noticeable for laterally extending images, such as a bridge,highway crosswalk markings, the earth's horizon, and the like. Suchimages are at different vertical angles with respect to the imagecapture devices. This results in different vertical positions on thedisplay causing the elongated image to appear disjointed.

A camera system provides a monocular view of the scene, compared to thebinocular, or stereoscopic, view obtained when the scene is viewedthrough a rearview mirror. This makes the ability to judge distances ina camera system a problem. This effect is most noticeable at distancesclose to the vehicle where stereoscopic imaging is relied uponextensively by the driver in judging relative locations of objects.Therefore, known camera systems fail to provide to the driver importantinformation where that information is most needed—at small separationdistances from surrounding objects.

Another difficulty with camera systems is that, in order to provide asufficient amount of information, the camera system typically presentsthe driver with a greatly increased field of view. This improvesperformance by further reducing blind spots at the side and rear of thevehicle. However, an increased field of view is often obtained byutilizing a wide-angle lens which introduces distortion of the scene andfurther impairs the ability of the driver to judge distances of objectsdisplayed. The problem with such distortion of the scene is that thedriver must concentrate more on the display and take a longer time tointerpret and extract the necessary information. This further distractsthe driver from the primary visual task of maintaining awareness ofvehicles and other objects in the vicinity of the driven vehicle.

SUMMARY OF THE INVENTION

The present invention is directed towards enhancing the interpretationof visual information in a rearview vision system by presentinginformation in a manner which does not require significant concentrationof the driver or present distractions to the driver. This isaccomplished according to the invention in a rearview vision systemhaving at least two image capture devices positioned on the vehicle anddirected rearwardly with respect to the direction of travel of thevehicle. A display is provided for images captured by the image capturedevices. The display combines the captured images into an image thatwould be achieved by a single rearward-looking camera having a viewunobstructed by the vehicle. In order to obtain all of the necessaryinformation of activity, not only behind but also along side of thevehicle, the virtual camera should be positioned forward of the driver.The image synthesized from the multiple image capture devices may have adead space which corresponds with the area occupied by the vehicle. Thisdead space is useable by the driver's sense of perspective in judgingthe location of vehicles behind and along side of the equipped vehicle.

The present invention provides techniques for synthesizing imagescaptured by individual, spatially separated, image capture devices intosuch ideal image, displayed on the display device. This may beaccomplished according to an aspect of the invention by providing atleast three image capture devices. At least two of the image capturedevices are side image capture devices mounted on opposite sides of thevehicle. At least one of the image capture devices is a center imagecapture device mounted laterally between the side image capture devices.A display system displays an image synthesized from outputs of the imagecapture devices. The displayed image includes an image portion from eachof the image capture devices. The image portion from the center imagecapture device is vertically compressed.

It has been discovered that such vertical compression substantiallyeliminates distortion resulting from the spatial separation between thecameras and can be readily accomplished. In an illustrated embodiment,the image compression is carried out by removing selective ones of thescan lines making up the image portion. A greater number of lines areremoved further away from the vertical center of the image.

The compression of the central image portion produces a dead space inthe displayed image which may be made to correspond with the area thatwould be occupied by the vehicle in the view from the single virtualcamera. Preferably, perspective lines are included at lateral edges ofthe dead space which are aligned with the direction of travel of thevehicle and, therefore, appear in parallel with lane markings. Thisprovides visual clues to the driver's sense of perspective in order toassist in judging distances of objects around the vehicle.

According to another aspect of the invention, image enhancement meansare provided for enhancing the displayed image. Such means may be in theform of graphic overlays superimposed on the displayed image. Suchgraphic overlap may include indicia of the anticipated path of travel ofthe vehicle which is useful in assisting the driver in guiding thevehicle in reverse directions. Such graphic overlay may include adistance grid indicating distances behind the vehicle of objectsjuxtaposed with the grid.

These and other objects, advantages, and features of this invention willbecome apparent by review of the following specification in conjunctionwith the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a top plan view of a vehicle having a rearview vision systemaccording to the invention;

FIG. 2 is a side elevation of the vehicle in FIG. 1;

FIG. 3 is a front elevation of a display according to the invention;

FIG. 4 is the same view as FIG. 1 illustrating an alternative embodimentof the invention;

FIG. 5 is a block diagram of an electronic system according to theinvention;

FIG. 6 is the same view as FIG. 3 illustrating an alternate mode ofoperation of the system;

FIG. 7 is the same view as FIG. 2 illustrating an alternative embodimentof the invention;

FIG. 8 is the same view as FIG. 3 illustrating an alternative embodimentof the invention;

FIG. 9 is the same view as FIGS. 1 and 4 illustrating an alternativeembodiment of the invention;

FIG. 10 is the same view as FIGS. 3 and 8 illustrating an alternativeembodiment of the invention; and

FIG. 11 is a chart illustrating the horizontal row of pixels (n1, n2) onwhich an object will be imaged from two longitudinally separated imagecapture devices as that object is spaced at different longitudinaldistances from the image capture devices.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring now specifically to the drawings, and the illustrativeembodiments depicted therein, a vehicle 10, which may be an automobile,a light truck, a sport utility vehicle, a van, a bus, a large truck, orthe like includes a rearview vision system, generally illustrated at 12,for providing a driver of the vehicle with a view rearwardly of thevehicle with respect to the direction of travel D of the vehicle (FIG.1). Vision system 12 includes at least two side image capture devices 14positioned, respectively, on opposite sides of vehicle 10 and a centerimage capture device 16 positioned on the lateral centerline of thevehicle. All of the image capture devices are directed generallyrearwardly of the vehicle. Rearview vision system 12 additionallyincludes an image processor 18 for receiving data signals from imagecapture devices 14, 16 and synthesizing, from the data signals, acomposite image 42 which is displayed on a display 20.

As will be set forth in more detail below, the images captured by imagecapture devices 14, 16 are juxtaposed on display 20 by image processor18 in a manner which approximates the view from a single virtual imagecapture device positioned forwardly of the vehicle at a location C andfacing rearwardly of the vehicle, with the vehicle being transparent tothe view of the virtual image capture device. Vision system 12 providesa substantially seamless panoramic view rearwardly of the vehiclewithout duplicate or redundant images of objects. Furthermore,elongated, laterally-extending, objects, such as the earth's horizon,appear uniform and straight across the entire displayed image. Thedisplayed image provides a sense of perspective, which enhances theability of the driver to judge location and speed of adjacent trailingvehicles.

Each of side image capture devices 14 has a field of view 22 and isaimed rearwardly with respect to the vehicle about an axis 24 which isat an angle, with respect to the vehicle, that is half of the horizontalfield of view of the image capture device. In this manner, each of theimage capture devices 14 covers an area bounded by the side of thevehicle and extending outwardly at an angle defined by the horizontalfield of view of the respective side image capture device. Center imagecapture device 16 has a horizontal field of view 26, which issymmetrical about the longitudinal axis of the vehicle. The field ofview of each side image capture device 14 intersect the field of view ofcenter image capture device 16 at a point P which is located a distanceQ behind vehicle 10.

Rear blind zones 30 are located symmetrically behind vehicle 10extending from the rear of the vehicle to point P. Side blind zones 25located laterally on respective sides of the vehicle extend rearwardlyof the forward field of view 36 of the driver to the field of view 22 ofthe respective side image capture device 14. An object will not becaptured by side image capture devices 14 or center image capturedevices 16 if the object is entirely within one of the blind zones 25,30. In order for an object, such as another vehicle V or other road usertravelling to the side of vehicle 10, to be observed by an operator ofvehicle 10, the object must be either at least partially within theforward field of view 36 of the driver or be captured by image capturedevices 14, 16 and displayed on display 20. FIG. 4 illustrates vehicle10 travelling on a three-lane highway having lanes L1, L2, and L3 withthe vehicle in lane L2. Another vehicle V is shown positioned mostlywithin one of the blind zones 25, but with the rearmost portion of thevehicle V extending into field of view 22 where the vehicle image willbe captured by one of side image capture devices 14. In the illustratedembodiment, vehicle V is a motorcycle travelling in the center of lanesL1 or L3 and represents a worst case for observing a vehicle travellingat least partially within one of the blind zones 25. In order for aportion of vehicle V to be extending either forwardly or rearwardly ofthe respective blind zone 25, where the vehicle V may be observed byeither the forward field of view 36 of the driver or by the rearviewvision system 12, the field of view 22 of side image capture devices 14must be sufficiently wide to capture a portion of vehicle V asillustrated in FIG. 4. Preferably, the horizontal field of view 22 ofside image capture devices 14 is no greater than that required toprovide sufficient coverage which would be in the range of betweenapproximately 55 degrees and approximately 70 degrees. In theillustrated embodiment, the horizontal field of view 22 is 61 degrees.In order for a portion of vehicle V to be within a vertical field ofview 40 of one of side image capture devices 14, the field of viewshould extend to the pavement at a plane M which intersects vehicle V(FIG. 2). Preferably, vertical field of view 40 is between approximately60 degrees and approximately 75 degrees. In the illustrated embodiment,vertical field of view 40 is 66 degrees.

A left overlap zone 32 and a right overlap zone 34 extend rearward fromrespective points P where the horizontal fields of view of the sideimage capture devices intersect the field of view of center imagecapture device 16. Overlap zones 32, 34 define areas within which anobject will be captured both by center image capture device 16 and oneof the side image capture devices 14. An object in an overlap zone 32,34 will appear on display 20 in multiple image portions in a redundantor duplicative fashion. In order to avoid the presentation of redundantinformation to the driver, and thereby avoid confusion and simplify thetask of extracting information from the multiple images or combinedimages on display 20, the object should avoid overlapping zones 32, 34.In practice, this may be accomplished to a satisfactory extent by movingpoints P away from the vehicle and thereby increasing distance Q. It isdesirable to increase distance Q to a length that will exclude vehiclestravelling at a typical separation distance behind vehicle 10 fromoverlapping zones 32, 34. This separation distance is usually a functionof the speed at which the vehicles on the highway are travelling. Thefaster the vehicles are travelling, the further Q should be moved behindvehicle 10 to keep overlap zones 32 and 34 outside of the recommendedvehicle spacing. If, however, the vehicles are travelling at a slowerspeed, then the generally accepted recommendation for vehicle spacingdecreases and it is more likely that a vehicle will be within overlapzone 32, 34. Therefore, the distance Q may be selected to accommodateexpected vehicle spacing for an average driving speed of vehicle 10.

Distance Q is a function of the effective horizontal field of view 26 ofcenter image capture device 16. As field of view 26 decreases, points Pmove further rearward of the vehicle from a distance Q₁, to a distanceQ₂, as best illustrated in FIG. 4. In order to increase distance Q toeliminate redundant and duplicative information displayed on display 20for most driving conditions of vehicle 10, field of view 26 ispreferably less than 12 degrees. In the illustrated embodiment, field ofview 26 is between 6 and 8 degrees. Alternatively, distance Q may bedynamically adjusted according to some parameter, such as the speed ofvehicle 10. This would allow Q to be greater when the vehicle istravelling at a faster speed, where vehicle separation tends to belarger, and vice versa. Field of view 26 may be adjusted by utilizing aselective presentation of pixels of the captured image in the displayedimage.

Referring to FIG. 3, image display device 20 displays a composite image42 made up of a left image portion 44, a right image portion 46, and acenter image portion 48. Each image portion 44-48 is reversed from theimage as captured by the respective image capture device 14, 16utilizing conventional techniques. These techniques include reading theimage in reverse with the image capture device, writing the image inreverse to display device 20, or reversing the image in image processor18. Left image portion 44 is joined with central image portion 48 at aboundary 50. Central image portion 48 is joined with right image portion46 at a boundary 52. As may best be seen in FIG. 3, the image portionsat boundaries 50 and 52 are continuous whereby composite image 42 is aseamless panoramic view rearwardly of the vehicle. As also is apparentfrom FIG. 3, central image portion 48 is narrower than either left imageportion 44 or right image portion 46. This is a result of reducing thehorizontal field of view 26 of center image capture device 16sufficiently to move points P, and thus overlap zones 32 and 34, asufficient distance behind vehicle 10 to reduce redundant andduplicative images between image portions 44-48. Composite image 42provides a clear image, which avoids confusion and simplifies the taskof extracting information from the multiple image portions 44-48. Asalso may be seen by reference to FIG. 3, display 20 may additionallyinclude indicia such as the readout of a compass 54, vehicle speed 56,turn signals 58, and the like as well as other graphical or videodisplays, such as a navigation display, a map display, and aforward-facing vision system. In this manner, rearview vision system 12may be a compass vision system or an information vision system.

In the embodiment of rearview vision system 12 having a dynamicallyadjusted value of distance Q, the spacing between boundaries 50 and 52will dynamically adjust in sequence with the adjustment of distance Q.Thus, as overlap zones 32, 34 move further away from the vehicle; forexample, in response to an increase in speed of the vehicle, boundarylines 50 and 52 will move closer together and vice versa. In thismanner, composite image 42 is dynamic, having image portions ofdynamically adaptive sizes.

Display 20 is of a size to be as natural as possible to the driver. Thisis a function of the size of the display and the distance between thedisplay and the driver. Preferably, the displayed image simulates animage reflected by a rearview mirror. As such, the size of display 20 isapproximately the combined areas of the three rearview mirrors (oneinterior mirror and two exterior mirrors) conventionally used withvehicles. As best seen by reference to FIG. 2, display 20 is preferablypositioned within the driver's physiological field of view withoutobstructing the view through the windshield. It is known that thedriver's field of view, with the head and eyes fixed forward, extendsfurther in a downward direction than in an upward direction. Display 20could be located above the vertical view through the windshield whereinthe display may be observed at the upward portion of the driver's fieldof view. However, the position for the display illustrated in FIG. 2 ispreferred wherein the display is within the lower portion of thedriver's field of view.

Display 20, in the illustrated embodiment, is a flat panel display, suchas a back-lit liquid crystal display, a plasma display, a field emissiondisplay, or a cathode ray tube. However, the synthesized image could bedisplayed using other display techniques such as to provide a projectedor virtual image. One such virtual display is a heads-up display. Thedisplay may be mounted/attached to the dashboard, facia or header, or tothe windshield at a position conventionally occupied by an interiorrearview mirror.

Although various camera devices may be utilized for image capturedevices 14, 16, an electro-optic, pixelated imaging array, located inthe focal plane of an optical system, is preferred. Such imaging arrayallows the number of pixels to be selected to meet the requirements ofrearview vision system 12. The pixel requirements are related to theimaging aspect ratio of the respective image capture devices, which, inturn, are a function of the ratio of the vertical-to-horizontal field ofview of the devices, as is well known in the art. In the illustratedembodiment, the imaging aspect ratio of side image capture devices 14 is2:1 and the image aspect ratio of central image capture device 16 isvariable down to 0.1:1. Such aspect ratio will produce images which willnot typically match that of commercially available displays. Acommercially available display may be used, however, by leaving ahorizontal band of the display for displaying alpha-numeric data, suchas portions of an instrument cluster, compass display, or the like, asillustrated in FIG. 3.

In the illustrated embodiment, image capture devices 14, 16 are CMOSimaging arrays of the type manufactured by VLSI Vision Ltd. ofEdinburgh, Scotland, which are described in more detail in co-pendingU.S. patent application Ser. No. 08/023,918 filed Feb. 26, 1993, byKenneth Schofield and Mark Larson for an AUTOMATIC REARVIEW MIRRORSYSTEM USING A PHOTOSENSOR ARRAY, the disclosure of which is herebyincorporated herein by reference. However, other pixelated focal planeimage-array devices, which are sensitive to visible or invisibleelectromagnetic radiation, could be used. The devices could be sensitiveto either color or monochromatic visible radiation or near or farinfrared radiation of the type used in night-vision systems. Each imagecapture device could be a combination of different types of devices,such as one sensitive to visible radiation combined with one sensitiveto infrared radiation. Examples of other devices known in the artinclude charge couple devices and the like.

Preferably, image capture devices 14 and 16 are all mounted at the samevertical height on vehicle 10, although compromise may be required inorder to accommodate styling features of the vehicle. The horizontal aimof image capture devices 14 and 16 is preferably horizontal. However,the portion of the image displayed is preferably biased toward thedownward portion of the captured image because significantly less usefulinformation is obtained above the horizontal position of the imagecapture devices.

Each image-capturing device 14, 16 is controlled by appropriatesupporting electronics (not shown) located in the vicinity of theimaging array such that, when operating power is supplied, either ananalog or a digital data stream is generated on an output signal linesupplied to image processor 18. The support electronics may be providedpartially on the image chip and partially on associated electronicdevices. For each exposure period, a value indicative of the quantity oflight incident on each pixel of the imaging array during the exposureperiod is sequentially outputted in a predetermined sequence, typicallyrow-by-row. The sequence may conform to video signal standards whichsupport a direct view such that, when a scene is viewed by animage-capturing device, the image presented on a display representsdirectly the scene viewed by the image-capturing devices. However, whenlooking forward and observing a displayed image of a rearward scene, thedriver will interpret the image as if it were a reflection of the sceneas viewed through a mirror. Objects to the left and rearward of thevehicle, as viewed by the rearward-looking camera, are presented on theleft-hand side of the display and vice versa. If this reversal iseffected in image processor 18, it may be by the use of a data storagedevice, or buffer, capable of storing all of the pixel values from oneexposure period. The data is read out of the data storage device in areversed row sequence. Alternatively, the imaging array electronicscould be constructed to provide the above-described reversal at theimage-capturing device or at the display.

Data transmission between image capture devices 14, 16 and imageprocessor 18 and/or between image processor 18 and display 20 may be byelectrically conductive leads or fiber-optic cable. It is possible, forparticular applications, to eliminate image processor 18 and directdrive display 20 from image capture devices 14, 16 at the pixel level.

The data streams from image-capturing devices 14, 16 are combined inimage processor 18 and directly mapped to the pixel array of display 20.This process is repeated preferably at a rate of at least 30 times persecond in order to present an essentially real time video image. Theimage captured by side image capture device 14 on the right side of thevehicle is presented in right image portion 46 and the image from sideimage capture device 14 on the left side of the vehicle is displayed onleft image portion 44. The image from center image capture device 16 isdisplayed on central image portion 48. The three image portions 44-48are presented in horizontal alignment and adjacent to each other.However, the composite image may be positioned at any desired verticalposition in the display 20. It is also possible to display imageportions 44-48 on separate image devices which are adjacent each other.

In vision system 12, side image capture devices 14 are positionedpreferably at a forward longitudinal position on vehicle 10 and centerimage capture device 16 is positioned at a rearward longitudinalposition on the vehicle. As best seen by reference to FIG. 7, thispositioning creates a difference in the vertical angle between each sideimage capture device 14 and center image capture device 16 with respectto a fixed location P₁ that is a distance D₁ behind the vehicle. Thisdifference in sensing angle will cause each side image capture device 14to image an object located at P₁ on a horizontal row of pixels that isdifferent from the horizontal row of pixels that center image capturedevice 16 will image the same object. If the image is below thehorizontal centerline of the image capture device, it will be imaged ona lower row of pixels by center image capture device 16 than the row ofpixels it will be imaged by the side image capture devices 14, asillustrated in FIG. 7. This mismatch between horizontal pixel rows ofthe captured image is furthermore a function of the distance of thecaptured image from the rear of the vehicle. This can be understood byreference to FIG. 11 which presents a chart 90 having a first column 92of pixel lines n1, measured from the array centerline, at which anobject will be imaged by side image capture device 14 and a secondcolumn 94 of pixel lines n2, measured from the array verticalcenterline, at which the same object will be imaged by center imagecapture device 16. The result is that an object, which is captured byboth side and center image capture devices 14, 16, will be verticallydisjointed at the boundary of the displayed image, if the object iscaptured by more than one image capture device. The amount ofdisjointment will be greater closer to the vehicle and less at furtherdistances. If the object is elongated in the horizontal direction, suchas earth's horizon, bridges, or cross-markings on highways, then theobject will appear to be either broken or crooked.

In order to provide uniform display of laterally elongated images, arearview vision system 12′ is provided having a central image portion48′ which is processed differently from the image display portions 44′and 46′ produced by the side image capture devices (FIG. 8). Centralimage portion 48′ is reduced vertically, or compressed, by removingspecified scan lines, or pixel rows, from the image captured by centerimage capture device 16 in a graduated fashion. The difference in thepixel line at which an object will be imaged by each of the side andcenter image capture devices is a function of the distance D of theobject from the rear of the vehicle, with a greater variation occurringat shorter distances and the variation reducing to zero for infinitedistances. Therefore, the compression of central image portion 48′ isnon-linear, with substantially no compression at the vertical center ofthe image and greater compression at greater distances above and belowthe vertical center point of the image. This is accomplished by removingspecific lines from the center display in a graduated fashion with agreater number of lines removed further from the vertical center of theimage. The removed lines may be merely discarded in order to verticallyreduce the image. Alternatively, the data contained in the removed linesmay be utilized to modify the value of adjacent pixels above and belowthe removed line in order to enhance the quality of the compressedimage. Averaging, median filtering, or other such known techniques mayalso be used.

Each of right image portion 46′ and left image portion 44′ includes anupper portion 64 which extends above the compressed upper portion of thecentral image portion 48′. In the illustrated embodiment, upper portions64 are deleted in order to present a uniform upper horizontal boundaryfor display 20′. In the illustrated embodiment, the mismatch between thelower horizontal boundary of central image portion 48′ and each of theleft and right image portions provides a dead space 66 which provides avisual prompt to the user of the approximate location of the rearwardcorners S of vehicle 10. This dead space 66 in the image displayed ondisplay 20′ approximates the footprint occupied by vehicle 10 whenviewed from point C. This is particularly useful because it provides avisual indication to the driver that a vehicle passing vehicle 10, asviewed in either left image portion 44′ or right image portion 46′, isat least partially adjacent vehicle 10 if the image of the approachingvehicle is partially adjacent to dead space 66.

In an alternative embodiment, the vertical compression technique may beapplied to only a lower vertical portion of central image portion 48′.In most driving situations, objects imaged by rearward-facing imagecapture devices above the horizon are at a long distance from thevehicle while those below the horizon get progressively closer to thevehicle in relation to the distance below the horizon in the displayedimage. Therefore, compression of the upper vertical portion of thecentral image portion may be eliminated without significant reduction inperformance.

Compression of the central image portion may also advantageously beprovided horizontally, as well as vertically. Spatial separation ofcenter image capture device 16 from side image capture devices 14 causessimilar distortion, as that described above, in the horizontaldirection. This effect is spherical in nature and would require a morecomplex corrective action, such as compressing the image based upon theremoval of pixels from an approximation to concentric circles centeredon the center of the imaging array, or other techniques which would beapparent to those skilled in the art.

A rearview vision system 12″ includes an image display 20″ having acompressed central image portion 48″ and left and right image portions44″ and 46″, respectively (FIG. 10). A border 50′ between left sideimage 44″ and central image 48″ includes a vertical central borderportion 50 a′, an upper border portion 50 b′, and a lower border portion50 c′. Upper border portion 50 b′ and lower border portion 50 c′ divergelaterally outwardly, vertically away from central portion 50 a′. Aborder 52′ between central image portion 48″ and right image portion 46″includes a central boundary portion 52 a′, an upper boundary portion 52b′, and a lower boundary portion 52 c′. Upper boundary portion 52 b′andlower boundary portion 52 c′diverge laterally outwardly vertically awayfrom central portion 52 a′. This creates an upper portion of centralimage portion 48″ and a lower portion of central image portion 48″ whichextend beyond the center portion thereof. This configuration is basedupon the realization that the surface of the road immediately behind thevehicle is captured by central image capture device 16. Likewise, thehorizontal plane above the vehicle, which is symmetrical with the roadsurface, is captured by the center image capture device. This may beseen by referring to point P in FIG. 10, which illustrate the pointswhere the effective radius 68 of the virtual image capture deviceintersects dead zones 30 and by referring to point S in FIG. 10 whichillustrates the corners or the rear of the vehicle (S).

The image displayed on display 20″ includes a dead space 66′ havingdiverging lateral sides 68 a, 68 b. Diverging sides 68 a and 68 b areconfigured in order to extend in the direction of travel of vehicle 10which is parallel to lane markings of a highway on which vehicle 10 istravelling. This further enhances the visual perception of the driver byproviding a visual clue of the location of images appearing on display20″ with respect to the vehicle 10. Side portions 68 a, 68 b, in theillustrated embodiment, are natural extensions of lower boundaryportions 50 c′ and 52 c′ and extend from point S on each respective sideof the vehicle to point R, which represents the intersection of thelower extent of the vertical field of view 40 of each side image capturedevice 14 with the pavement (FIG. 7).

Rearview vision systems 12′ and 12″ utilize a displayed synthesizedimage which takes into account the use of perspective in enhancing thedriver's understanding of what is occurring in the area surrounding thevehicle. The images produced on displays 20′ and 20″ effectively removethe vehicle bodywork and replace the bodywork with a vehicle footprintas would be viewed by virtual camera C. The image displayed on display20″ further includes perspective lines which further enhance the roll ofperspective in the driver's understanding of what is occurring.

In order to further enhance the driver's understanding of what isoccurring in the area surrounding the vehicle, a rearview vision system12′″ includes a display 20′″ having image enhancements (FIG. 6). In theillustrative embodiment, such image enhancements include graphicoverlays 70 a, 70 b which are hash marks intended to illustrate to thedriver the anticipated path of movement of vehicle 10. In theillustrated embodiment, the anticipated vehicle motion is a function ofthe vehicle direction of travel as well as the rate of turn of thevehicle. The forward or rearward direction of vehicle travel isdetermined in response to the operator placing the gear selection device(not shown) in the reverse gear position. The degree of turn of thevehicle may be determined by monitoring the movement of the vehiclesteering system, monitoring the output of an electronic compass, ormonitoring the vehicle differential drive system. In the embodimentillustrated in FIG. 6, the configuration of graphic overlays 70 a, 70 bindicates that the vehicle is in reverse gear and that the wheels areturned in a manner that will cause the vehicle to travel toward thedriver's side of the vehicle. If the wheels were turned in the oppositedirection, graphic overlays 70 a, 70 b would curve clockwise toward theright as viewed in FIG. 6. If the vehicle's wheels were straight,graphic overlays 70 a, 70 b would be substantially straight converginglines. If the vehicle is not in reverse gear position, graphic overlays70 a, 70 b are not presented. Other types of graphic overlays of thedisplayed image are comprehended by the invention.

Horizontal grid markings on the display may be provided to indicatedistances behind the vehicle at particular markings. Such grid wouldallow the driver to judge the relative position of vehicles behind theequipped vehicle. In one embodiment, short horizontal lines aresuperimposed on the displayed image at regular rearward intervals inhorizontal positions which correspond to the boundaries of the lane inwhich the vehicle is travelling. In order to avoid confusion when thevehicle is travelling in a curved path, from a lack of correspondencebetween the graphic overlay and the road, a signal indicative of thevehicle's rate of turn may be taken into account when generating thegraphic overlay. In this manner, the distance indications may be movedlaterally, with reduced horizontal separation, to correspond to thepositions of the curved lane boundaries and vertically on the image tocompensate for the difference between distances along a straight andcurved path.

Another image enhancement is to alter the appearance of an object in aparticular zone surrounding the vehicle in order to provide anindication, such as a warning, to the driver. As an example, a vehiclethat is too close to the equipped vehicle for safe-lane change, may bedisplayed in a particular color, such as red, may flash, or otherwise bedistinguishable from other images on the display. Preferably, the speedof the equipped vehicle 10, which may be obtained from known speedtransducers, may be provided as an input to the rearview vision systemin order to cause such warning to be a function of the vehicle speedwhich, in turn, affects the safe separation distance of vehicles. Theoperation of the turn signal may also be used to activate suchhighlighting of other road users or to modify the scope of the imagedisplayed. In order to determine the distance of objects behind vehicle10, a separate distance-measuring system may be used. Such separatesystem may include radar, ultrasonic sensing, infrared detection, andother known distance-measuring systems. Alternatively, stereoscopicdistance-sensing capabilities of side image capture devices 14 may beutilized to determine the separation distance from trailing objectsutilizing known techniques.

Thus, it is seen that the image displayed on display 20-20′″ may bedifferent under different circumstances. Such different circumstancesmay relate to the vehicle's direction of travel, speed, rate of turn,separation from adjacent objects, and the like.

Various other forms of image processing may be utilized with rearviewvision system 12-12′″. Luminant and chrominant blending may be appliedto the images captured by image capture devices 14, 16 in order toproduce equality of the image data whereby the image portions appear asif they were produced by one image capture device. The dynamic range ofthe image capture devices may be extended in order to provide highquality images under all lighting conditions. Furthermore, individualpixel groups may be controlled in order to selectively compensate forbright or dark spots. For example, anti-blooming techniques may beapplied for bright spots. Multiple exposure techniques may be applied tohighlight dark areas. Image morphing and warping compensation techniquesmay additionally be applied. Resolution of the image capture devices anddisplay may be selected in order to provide sufficient image quality forthe particular application.

A heater may be applied to each image capture device in order to removedew and frost that may collect on the optics of the device. Although, inthe illustrative embodiment, the optical centerline of the cameracoincides with the field of view, particular applications may result inthe centerline of the camera pointing in a direction other than thecenterline of the field of view. Although, in the illustrativeembodiment, the image capture devices are fixed, it may be desirable toprovide selective adjustability to the image capture devices or opticalpaths in particular applications. This is particularly desirable whenthe system is used on articulated vehicles where automated andcoordinated camera aim may be utilized to maintain completeness of thesynthesized image.

When operating the vehicle in the reverse direction, it may be desirableto provide additional data concerning the area surrounding the immediaterear of the vehicle. This may be accomplished by utilizingnon-symmetrical optics for the center image capture device in order toprovide a wide angle view at a lower portion of the field of view.Alternatively, a wide angle optical system could be utilized with theelectronic system selectively correcting distortion of the capturedimage. Such system would provide a distortion-free image while obtainingmore data, particularly in the area surrounding the back of the vehicle.

The invention additionally comprehends the use of more than three imagecapture devices. In addition to side image capture devices positioned atthe front sides of the vehicle and a center image capture devicepositioned at the center rear of the vehicle, additional image capturedevices may be useful at the rear corners of the vehicle in order tofurther eliminate blind spots. It may additionally be desirable toprovide an additional center image capture device at a higher elevationin order to obtain data immediately behind the vehicle and thereby fillin the road surface detail immediately behind the vehicle. Suchadditional detail is particularly useful when operating the vehicle inthe reverse direction. Of course, each of the image capture devicescould be a combination of two or more image capture devices.

Although the present invention is illustrated as used in a rearviewvision system, it may find utility in other applications. For example,the invention may be useful for providing security surveillance in anarea where a building or other object obstructs the view of the areaunder surveillance. Additionally, the invention may find application innight-vision systems and the like. For example, the invention may beapplied to forward-facing night-vision systems, or other visionenhancement systems such as may be used in adverse weather oratmospheric conditions such as fog, applied to provide an enhanceddisplay of a synthesized image, which approximates a forward-facing viewfrom a single virtual camera located rearwardly of the driver, takingadvantage of the perspective features of the image.

Thus, it is seen that the present invention enhances the relationshipbetween the driver's primary view and the image presented on therearview vision system. This is accomplished in a manner which providesease of interpretation while avoiding confusion so that the driver doesnot have to concentrate or look closely at the image. In this manner,information presented on the display is naturally assimilated. This isaccomplished while reducing blind spots so that other vehicles orobjects of interest to the driver will likely be displayed to thedriver. Additionally, the use of perspective allows distances to be moreaccurately determined.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the invention,which is intended to be limited only by the scope of the appendedclaims, as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

The embodiments of the invention in which an exclusive property orprivilege is claimed are defined as follows:
 1. A rearview vision systemfor a vehicle having a gear actuator, comprising: an image capturedevice mounted at the rear of the vehicle and having a field of viewdirected rearwardly of the vehicle; a display system viewable by adriver of the vehicle which displays a rearward image output of saidimage capture device; a graphic overlayer superimposed on said rearwardimage when the gear actuator of the vehicle selects a reverse gear; andwherein said graphic overlayer includes indicia of the anticipated pathof travel of the vehicle.
 2. The rearview system of claim 1 wherein saidindicia responds to the direction of turn of the vehicle.
 3. Therearview system of claim 1 wherein said indicia responds to at least oneof the vehicle's steering system, the vehicle's differential system anda compass.
 4. The rearview system of claim 1 wherein said graphicoverlayer is not superimposed on said rearward image when the gearactuator of the vehicle does not select a reverse gear.
 5. The rearviewsystem of claim 1 wherein said indicia comprise markings that provideindication of distance behind the vehicle.
 6. The rearview system ofclaim 1 wherein said indicia comprise markings superimposed at intervalson said rearward image corresponding to the boundaries of the lane inwhich the vehicle is reversing.
 7. The rearview system of claim 6wherein said graphic overlayer is responsive to a signal indicative ofthe rate of turn of the vehicle.
 8. The rearview system of claim 1wherein said graphic overlayer is responsive to a signal indicative ofthe rate of turn of the vehicle.
 9. The rearview system of claim 1wherein said graphic overlayer comprises markings that move laterally onsaid rearward image, with reduced separation, to correspond to positionsof a curved lane boundary and vertically on said rearward image tocompensate for the difference between distances along a straight andcurved path.
 10. The rearview system of claim 1 wherein said rearviewvision system includes a distance-sensing system.
 11. The rearviewsystem of claim 10 wherein said distance-sensing system is selected fromthe group consisting of a radar, an ultrasonic sensing, and an infrareddetection distance-measuring system.
 12. The rearview system of claim 1wherein said image capture device has a field of view which issymmetrical about the longitudinal axis of the vehicle.
 13. The rearviewsystem of claim 1 wherein said image capture device comprises apixilated imaging array.
 14. The rearview system of claim 13 whereinsaid pixilated array comprises a CMOS imaging array.
 15. The rearviewsystem of claim 1 wherein said graphic overlayer has a form that is afunction of at least one of the direction of travel and speed of thevehicle.
 16. The rearview system of claim 15 wherein said indiciacomprises at least one mark superimposed on said rearward image.
 17. Therearview system of claim 1 wherein said rear vision system includes amonitoring device for monitoring vehicle turning.
 18. The rearviewsystem of claim 17 wherein said monitoring device comprises one of amonitor of movement of the vehicle's steering system, a monitor of anoutput of an electronic compass and a monitor of the vehicle'sdifferential drive system.
 19. The rearview system of claim 1 whereinsaid display system comprises one of a flat panel display and a cathoderay tube.
 20. The rearview system of claim 1 wherein said display systemcomprises a flat panel display.
 21. The rearview system of claim 20wherein said flat panel display comprises one of a liquid crystaldisplay, a plasma display and a field emission display.
 22. The rearviewsystem of claim 20 wherein said flat panel display comprises a liquidcrystal display.
 23. The rearview system of claim 1 wherein said displaysystem is positioned within the field of view of the driver withoutobstructing the view through the windshield.
 24. The rearview system ofclaim 1 wherein said display system is mounted to one of the dashboard,facia, header and windshield of the vehicle.
 25. The rearview system ofclaim 1 wherein said display system is mounted at a positionconventionally occupied by an interior rearview mirror.
 26. The rearviewsystem of claim 1 wherein said display system comprises a display of oneof a projected and a virtual image.
 27. The rearview system of claim 1wherein said display system comprises a heads-up display.
 28. Therearview system of claim 1 wherein said indicia comprises at least onemark superimposed on said rearward image.
 29. The rearview system ofclaim 28 wherein said at least one mark superimposed on said rearwardimage comprises a plurality of marks superimposed on said rearward imageat rearward intervals.
 30. The rearview system of claim 29 wherein saidplurality of marks superimposed on said rearward image are positioned tocorrespond to boundaries of the lane in which the vehicle is traveling.31. The rearview system of claim 29 wherein said plurality of marks aremoved laterally to correspond to positions of curved lane boundarieswhen the vehicle is turning.
 32. A rearview vision system for a vehiclehaving a gear actuator, comprising: an image capture device mounted atthe rear of the vehicle and having a field of view directed rearwardlyof the vehicle; a display system viewable by a driver of the vehiclewhich displays a rearward image output of said image capture device; agraphic overlayer superimposed on said rearward image when the gearactuator of the vehicle selects a reverse gear; and wherein said graphicoverlayer is disabled when the gear actuator of the vehicle is not inreverse gear.
 33. The rearview system of claim 32 wherein said graphicoverlayer includes indicia of the anticipated path of travel of thevehicle.
 34. The rearview system of claim 33 wherein said indiciaresponds to at least one of the vehicle's steering system, the vehicle'sdifferential system and a compass.
 35. The rearview system of claim 33wherein said indicia responds to the direction of turn of the vehicle.36. The rearview system of claim 33 wherein said indicia comprisemarkings that provide indication of distance behind the vehicle.
 37. Therearview system of claim 33 wherein said indicia comprise markingssuperimposed at intervals on said rearward image corresponding to theboundaries of the lane in which the vehicle is reversing.
 38. Therearview system of claim 37 wherein said graphic overlayer is responsiveto a signal indicative of the rate of turn of the vehicle.
 39. Therearview system of claim 32 wherein said graphic overlayer is responsiveto a signal indicative of the rate of turn of the vehicle.
 40. Therearview system of claim 32 wherein said indicia comprise markings thatmove laterally on said rearward image, with reduced separation, tocorrespond to positions of a curved lane boundary and that movevertically on said rearward image to compensate for the differencebetween distances along a straight and curved path.
 41. The rearviewsystem of claim 32 wherein said rearview vision system includes adistance-sensing system.
 42. The rearview system of claim 41 whereinsaid distance-sensing system is selected from the group consisting of aradar, an ultrasonic sensing, and an infrared detectiondistance-measuring system.
 43. The rearview system of claim 32 whereinsaid image capture device has a field of view which is symmetrical aboutthe longitudinal axis of the vehicle.
 44. The rearview system of claim32 wherein said image capture device comprises a pixilated imagingarray.
 45. The rearview system of claim 44 wherein said pixilated arraycomprises a CMOS imaging array.
 46. The rearview system of claim 33wherein said graphic overlayer has a form that is a function of at leastone of the direction of travel and speed of the vehicle.
 47. Therearview system of claim 46 wherein said indicia comprises at least onemark superimposed on said rearward image.
 48. The rearview system ofclaim 32 wherein said rear vision system includes a monitoring devicefor monitoring vehicle turning.
 49. The rearview system of claim 48wherein said monitoring device comprises one of a monitor of movement ofthe vehicle's steering system, a monitor of an output of an electroniccompass and a monitor of the vehicle's differential drive system. 50.The rearview system of claim 32 wherein said display system comprisesone of a flat panel display and a cathode ray tube.
 51. The rearviewsystem of claim 32 wherein said display system comprises a flat paneldisplay.
 52. The rearview system of claim 51 wherein said flat paneldisplay comprises one of a liquid crystal display, a plasma display anda field emission display.
 53. The rearview system of claim 51 whereinsaid flat panel display comprises a liquid crystal display.
 54. Therearview system of claim 32 wherein said display system is positionedwithin the field of view of the driver without obstructing the viewthrough the windshield.
 55. The rearview system of claim 32 wherein saiddisplay system is mounted to one of the dashboard, facia, header andwindshield of the vehicle.
 56. The rearview system of claim 32 whereinsaid display system is mounted at a position conventionally occupied byan interior rearview mirror.
 57. The rearview system of claim 32 whereinsaid display system comprises a display of one of a projected and avirtual image.
 58. The rearview system of claim 32 wherein said displaysystem comprises a heads-up display.
 59. The rearview system of claim 33wherein said indicia comprises at least one mark superimposed on saidrearward image.
 60. The rearview system of claim 59 wherein said atleast one mark superimposed on said rearward image comprises a pluralityof marks superimposed on said rearward image at rearward intervals. 61.The rearview system of claim 60 wherein said plurality of markssuperimposed on said rearward image are positioned to correspond toboundaries of the lane in which the vehicle is traveling.
 62. Therearview system of claim 60 wherein said plurality of marks are movedlaterally to correspond to positions of curved lane boundaries when thevehicle is turning.
 63. A rearview vision system for a vehicle having agear actuator, comprising: an image capture device mounted at the rearof the vehicle and having a field of view directed rearwards of thevehicle; a display system viewable by a driver of the vehicle; saidimage capture device utilizing a wide angle optical system in order toprovide a wide angle view of an area rearwards of the vehicle whereinsaid wide angle optical system comprises non-symmetrical optics; andwherein said display system displays a rearward image output of saidimage capture device when the gear actuator of the vehicle selects areverse gear to operate the vehicle in reverse and wherein said displayof said rearward image output of said image capture device is disabledwhen the vehicle's gear actuator is not in reverse gear.
 64. Therearview system of claim 63 wherein said wide angle view comprises aview at a lower portion of said rearwards directed field of view of saidimage capture device.
 65. The rearview system of claim 64 wherein saidwide angle view comprises an area at the back of the vehicle.
 66. Therearview system of claim 65 wherein the output of said image capturedevice is corrected for distortion in the image captured by said wideangle optical system.
 67. The rearview system of claim 66 wherein saidcorrection of said distortion is achieved electronically.
 68. Therearview system of claim 65 wherein said rearview vision system includesa distance-measuring system.
 69. The rearview system of claim 68 whereinsaid distance-measuring system is selected from the group consisting ofa radar, an ultrasonic sensing, and an infrared detectiondistance-measuring system.
 70. The rearview system of claim 65 whereinsaid image capture device has a field of view which is symmetrical aboutthe longitudinal axis of the vehicle.
 71. The rearview system of claim65 wherein said image capture device comprises a pixelated imagingarray.
 72. The rearview system of claim 71 wherein said pixelated arraycomprises a CMOS imaging array.
 73. The rearview system of claim 65wherein said display system comprises one of a flat panel display and acathode ray tube.
 74. The rearview system of claim 65 wherein saiddisplay system comprises a flat panel display.
 75. The rearview systemof claim 74 wherein said flat panel display comprises one of a liquidcrystal display, a plasma display and a field emission display.
 76. Therearview system of claim 75 wherein said flat panel display comprises aliquid crystal display.
 77. The rearview system of claim 65 wherein saiddisplay system is positioned within the field of view of the driverwithout obstructing the view through the windshield.
 78. The rearviewsystem of claim 65 wherein said display system is mounted to one of thedashboard, facia, header and windshield of the vehicle.
 79. The rearviewsystem of claim 65 wherein said display system is mounted at a positionconventionally occupied by an interior rearview mirror.
 80. The rearviewsystem of claim 65 wherein said display system comprises display of oneof a projected and a virtual image.
 81. The rearview system of claim 65wherein said display system comprises a heads-up display.
 82. A rearviewvision system for a vehicle having a gear actuator, comprising: an imagecapture device mounted at the rear of the vehicle and having a field ofview directed rearwards of the vehicle; a display system viewable by adriver of the vehicle; said image capture device utilizing a wide angleoptical system comprising non-symmetrical optics in order to provide awide angle view of an area rearwards of the vehicle; and wherein saiddisplay system displays a rearward image output of said image capturedevice when the gear actuator of the vehicle selects a reverse gear tooperate the vehicle in reverse.
 83. The rearview system of claim 82wherein said wide angle view comprises a view at a lower portion of saidrearwards directed field of view of said image capture device.
 84. Therearview system of claim 83 wherein said wide angle view comprises anarea at the back of the vehicle.
 85. The rearview system of claim 82wherein said display of said rearward image output of said image capturedevice is disabled when the vehicle's gear actuator is not in reversegear.
 86. The rearview system of claim 82 wherein the output of saidimage capture device is corrected for distortion in the image capturedby said wide angle optical system.
 87. The rearview system of claim 86wherein said correction of said distortion is achieved electronically.88. The rearview system of claim 82 wherein said rearview vision systemincludes a distance-measuring system.
 89. The rearview system of claim88 wherein said distance-measuring system is selected from the groupconsisting of a radar, an ultrasonic sensing, and an infrared detectiondistance-measuring system.
 90. The rearview system of claim 82 whereinsaid image capture device has a field of view which is symmetrical aboutthe longitudinal axis of the vehicle.
 91. The rearview system of claim82 wherein said image capture device comprises a pixelated imagingarray.
 92. The rearview system of claim 91 wherein said pixelated arraycomprises a CMOS imaging array.
 93. The rearview system of claim 82wherein said display system comprises one of a flat panel display and acathode ray tube.
 94. The rearview system of claim 82 wherein saiddisplay system comprises a flat panel display.
 95. The rearview systemof claim 94 wherein said flat panel display comprises one of a liquidcrystal display, a plasma display and a field emission display.
 96. Therearview system of claim 95 wherein said flat panel display comprises aliquid crystal display.
 97. The rearview system of claim 82 wherein saiddisplay system is positioned within the field of view of the driverwithout obstructing the view through the windshield.
 98. The rearviewsystem of claim 82 wherein said display system is mounted to one of thedashboard, facia, header and windshield of the vehicle.
 99. The rearviewsystem of claim 82 wherein said display system is mounted at a positionconventionally occupied by an interior rearview mirror.
 100. Therearview system of claim 82 wherein said display system comprisesdisplay of one of a projected and a virtual image.
 101. The rearviewsystem of claim 82 wherein said display system comprises a heads-updisplay.